Created new desktop-messaging tab under messaging/desktop for desktop…#830
Open
Kand1Kane wants to merge 5 commits into
Open
Created new desktop-messaging tab under messaging/desktop for desktop…#830Kand1Kane wants to merge 5 commits into
Kand1Kane wants to merge 5 commits into
Conversation
… experiment Under messaging system, added new desktop messaging that includes omc experimenter information. Also, updated
Kand1Kane
requested review from
freshstrangemusic,
jaredlockhart,
mikewilli and
yashikakhurana
as code owners
Author
|
Here's the bug related to this PR: https://bugzilla.mozilla.org/show_bug.cgi?id=1997598 |
… experiment Under messaging system, added new desktop messaging that includes omc experimenter information. Also, updated
…rimenter-docs into desktop-messaging
Author
|
I've also requested a review from @mimizillaa to verify the documentation changes. Even if the code looks good from your end, please hold off on approving until we have their confirmation. I'll update this PR once we get their feedback — appreciate the patience! |
Comment on lines
+179
to
+180
| ```bash | ||
| "C:\Program Files\Mozilla Firefox\firefox.exe" --first-startup --profile "C:\Users\{username}\AppData\Roaming\Mozilla\Firefox\Profiles\{profileName}" |
Member
There was a problem hiding this comment.
This is not bash.
Additionally, you can use -P profileNameif it is registered in profiles.ini
| ```javascript title="user.js" | ||
| /* This Source Code Form is subject to the terms of the Mozilla Public | ||
| * License, v. 2.0. If a copy of the MPL was not distributed with this | ||
| * file, You can obtain one at http://mozilla.org/MPL/2.0/. */ |
Member
There was a problem hiding this comment.
I don't think this is copyrightable FWIW
Comment on lines
+131
to
+133
| ```javascript | ||
| await ChromeUtils.importESModule("resource://nimbus/lib/ExperimentManager.sys.mjs").ExperimentManager.generateTestIds((await (await fetch("https://firefox.settings.services.mozilla.com/v1/buckets/main-preview/collections/nimbuspreview/records/{YOUR_EXPERIMENT_SLUG}")).json()).data) | ||
| ``` |
Member
There was a problem hiding this comment.
Please update this to reference Nimbus DevTools
| "messaging-system.rsexperimentloader.collection_id", | ||
| "nimbus-preview" | ||
| ); | ||
| user_pref("app.normandy.user_id", "${GENERATED_BRANCH_NORMANY_ID_GOES_HERE}"); |
Member
There was a problem hiding this comment.
normandy ID is no longer used for bucketing. profile group IDs are and those are trickier to fake but the pref is "toolkit.telemetry.cachedProfileGroupID"
|
|
||
| :::tip | ||
|
|
||
| Use "Show Finder" (for root directory) to get quick access to the folder. Also save the path to the profile for the next step. |
Member
There was a problem hiding this comment.
Suggested change
| Use "Show Finder" (for root directory) to get quick access to the folder. Also save the path to the profile for the next step. | |
| Use "Show in Finder" (for root directory) to get quick access to the folder. Also save the path to the profile for the next step. |
Show in Finder is only on mac. On windows its "Open Folder" and I'm not sure about linux.
|
|
||
| When picking a feature ID for a non-holdback experiment, you should first prefer to run it on the [feature that matches the surface](/messaging/desktop/desktop-messaging-surfaces) that your experiment is using (`spotlight`, `featureCallout`, `infobar`, `cfr`, etc.). | ||
|
|
||
| If there are any other experiments using that feature ID that overlap in time with yours, you may still be able to use it, but ask in [#omc](https://mozilla.enterprise.slack.com/archives/C90HG2UQH) to verify. |
Member
There was a problem hiding this comment.
again don't link to slack.
|
|
||
| **Audience Overlap Considerations:** | ||
|
|
||
| In the case of experiment targeting overlaps, for each segment that overlaps, you must ensure that the sum of all audience sizes, including the audience size of your experiment is no more than 100%. For example, if an experiment exists that uses the same targeting segment as yours is enrolling at 30%, your experiment may use that same feature ID at up to 70%, for a combined 100% of the population of that targeting segment. |
Member
There was a problem hiding this comment.
Suggested change
| In the case of experiment targeting overlaps, for each segment that overlaps, you must ensure that the sum of all audience sizes, including the audience size of your experiment is no more than 100%. For example, if an experiment exists that uses the same targeting segment as yours is enrolling at 30%, your experiment may use that same feature ID at up to 70%, for a combined 100% of the population of that targeting segment. | |
| In the case of experiment targeting overlaps you must ensure that the sum of all audience sizes, including the audience size of your experiment, is no more than 100% for each segment that overlaps. For example, if an experiment exists that uses the same targeting segment as yours is enrolling at 30%, your experiment may use that same feature ID at up to 70%, for a combined 100% of the population of that targeting segment. |
|
|
||
| ## Testing the Experiment | ||
|
|
||
| When creating the JSON for treatment branches, you can verify the changes by updating the experiment to Preview Mode and by selecting **Launch to Preview**. This will provide a link that will force enroll the user into the experiment. Alternatively, you can copy paste the JSON into `about:asrouter` to preview it. |
Member
There was a problem hiding this comment.
Coming soon: enrolling via Nimbus devtools integrations into Experimenter :) But don't update your doc with this.
|
|
||
| Experiments take priority over rollouts, so one option is to share an ID with an existing rollout. If all IDs are taken by an experiment, it may still be usable depending on audience overlap. Check whether the targeting populations intersect. If your experiment targets Germany and the existing one targets Canada, they don't share users and can safely reuse the same ID. | ||
|
|
||
| The key is that the 'effective audience' (after all targeting criteria) doesn't overlap, not just that the IDs are "free." This also includes things like the channel, locales, and the enrollment %. |
Member
There was a problem hiding this comment.
Suggested change
| The key is that the 'effective audience' (after all targeting criteria) doesn't overlap, not just that the IDs are "free." This also includes things like the channel, locales, and the enrollment %. | |
| The key is that the 'effective audience' (after all targeting criteria) doesn't overlap, not just that the IDs are "free." This also includes things like the channel, locales, and the audience percentage. |
|
|
||
| When creating the JSON for treatment branches, you can verify the changes by updating the experiment to Preview Mode and by selecting **Launch to Preview**. This will provide a link that will force enroll the user into the experiment. Alternatively, you can copy paste the JSON into `about:asrouter` to preview it. | ||
|
|
||
| ### Previewing in `about:asrouter` |
Member
There was a problem hiding this comment.
Also coming soon, previewing messages with nimbus-devtools directly from experimenter :)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Because
A bunch of omc-experiment documentation and desktop messaging experiment feature/information are not mentioned in the current documentation.
This commit
Fixes #831